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FOREWORD 


ARI's  Leadership  Performance  Technical  Area  is  involved  in  research 
designed  to  enhance  the  effectiveness  of  Army  leadership  training, 
provide  new  and  improve  existing  procedures  and  instruments  to  assess 
leadership  potential,  provide  selection  and  assessment  technology  and 
base  data  for  use  in  assessment  systems  development,  and  to  provide 
Integrated  effort  on  the  Officer  Personnel  Management  System  (OPMS)  and 
Officer  Evaluation  System  Master  Management  Plan  (CESMMP)  problems. 

The  present  publication  is  concerned  with  OPMS  requirements  that  have 
been  identified  and  requested  by  the  Deputy  Chief  of  Staff  for  Personnel. 
These  requirements  have  been  integrated  with  an  expanded  research  program 
to  develop  the  officer  performance  data  base  to  meet  the  needs  of  person¬ 
nel  management. 

The  present  Research  Problem  Review  reports  on  research  that  has 
been  initiated  to  determine  the  potential  utilization  and  value  of  a 
peer  evaluation  program  for  senior  officers  for  use  in  identification 
of  promotion  potential,  nomination  for  high  level  military  schools. 
Command  designation,  and  special  assignments. 

The  entire  task  is  responsive  to  the  special  requirements  of  the 
Deputy  Chief  of  staff  for  Personnel  as  well  as  the  objectives  of  RDTE 
Project  2Q16310A755*  Manpower  Systems  Division. 


J.  E.  UHLANER 
Technical  Director 
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BRIEF 


Requirement : 


'?e~iie*6sa&ae  £tie  optimal  utilization  of  peer  nominations  among  senior 
officers *as  a  possible  technique  for  selection  for  promotion,  senior 


service  colleges,  duty  assignments,  and  periodic  assessments  for  use 
with  the  proposed  Officer  Personnel  Management  System  (OPMS). y 


Procedure: 

~^Peer  nominations  were  obtained,  by  mail,  from  two  independent  samples 
(a  total  of  17751  a  return  rate  of  76$)  of  active  duty  Army  colonels. 

The  instructions  required  that  the  rater  identify  the  officers  about 
whom  they  could  make  a  reasoned  judgment,  then  indicate  those  considered 
to  possess  the  potential  for  promotion  to  General  Officer.  Explanatory 
comments  concerning  the  basis  for  selection  or  any  aspect  of  the  rating 
procedure  were  Invited.  * 


Findings : 

The  peer  nominations  were  found  to  be  administratively  feasible,  but 
accepted  by  less  than  half  of  the  respondents.  The  most  acceptable  point 
of  application  was  among  the  more  junior  respondents  to  permit  the  use 
of  peer  ratings  for  senior  service  college  selection  and  possibly  promotion 
later.  The  spread  of  scores  indicated  that  the  technique  yields  adequate 
differentiation  and  reliability  for  operational  use  once  the  problem  of 
acceptability  is  solved,  j) 

Utilization  of  Findings: 

The  utility  of  peer  nominations  for  selection  for  promotion,  senior 
service  college,  duty  assignment,  and  periodic  assessment  has  been 
established.  In  order  to  obtain  maximum  utility  of  peer  nominations, 
overall  acceptability  of  the  groups  Involved  must  be  present  or  be 
obtained. 
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PURPOSE 

ARI's  Officer  Performance  Evaluation  Systems  work  unit  is  concerned 
with  development  of  improved  and  more  comprehensive  techniques  to  meet 
the  expanded  requirements  for  more  and  better  evaluative  information  on 
the  performance  and  potential  of  the  individual  officer.  This  information 
is  required  by  personnel  management  in  making  decisions  on  the  promotion, 
schooling,  and  duty  assignments  of  Army  officers. 

The  present  investigation,  requested  by  DCSPER(CSD)  (Deputy  Chief  of 
Staff  for  Personnel),  deals  with  the  utilization  of  peer  ratings  among 
senior  officer  groups  as  a  possible  technique  in  the  development  of  the 
proposed  Officer  Personnel  Management  Systems  (OPMS).  Specific  questions 
posed  by  the  DCSPER  request  concerned: 

1.  Feasibility  of  operational  administration. 

2.  Acceptability  of  the  peer  rating  technique  by  the  colonel  group. 

3.  Application  of  results: 

a.  For  promotion  to  general  officer 

(1)  Use  by  promotion  boards 

(2)  Use  by  Promotion  Approving  Authority  only 

b.  For  selection  for  senior  service  colleges 

c.  By  personnel  managers  in  making  duty  assignments 

d.  For  personnel  actions  In  lower  officer  grades,  or  at  regular 
calendar  periods  throughout  the  officer's  career. 

4.  Adequacy  of  differentiation--the  degree  to  which  sums  of  high 
nominations  minus  low  nominations  yielded  a  wide  range  of  scores. 


RESEARCH  DESIGN 

Plans  for  the  investigation  and  construction  of  the  experimental 
materials  and  procedures  for  field  administration  were  worked  out  in 
collaboration  and  review  with  the  DCSPER  sponsors.  These  plans  provided 
for  the  mail  administration  of  the  peer  rating  procedures  to  two  indepen¬ 
dent  samples  of  Army  colonels. 


SAMPLING 


Sample  1.  Officers,  still  on  active  duty,  selected  for  AUS-APL 
(Army  of  the  United  States  Army  promotion  list)  promotion  to  colonel, 

FY  66  (listed  on  DA  Circular  624-31  of  3  March  1965)  were  the  peer 
group,  with  the  members  rating  each  other,  a  total  of  321  officers. 

Officers,  still  on  active  duty,  selected  for  AUS-APL  promotion  to 
colonel,  FY  67  (listed  on  DA  Circular  624-6  of  1  March  1966),  also  made 
ratings  within  the  above  group  but  were  not  themselves  rated,  a  total 
of  439  officers. 

Sample  2.  Officers,  still  on  active  duty,  selected  for  AUS-APL 
promotion  to  colonel,  FY  69  ( listed  on  DA  Circular  624-61  of  6  November 
1968)  were  the  peer  group,  with  the  members  rating  each  other,  a  total 
of  886  officers. 

Officers,  still  on  active  duty,  selected  for  AUS-APL  promotion  to 
colonel,  FY  70  (listed  on  DA  Circular  624-82  of  3  October  1969)  also 
made  ratings  within  the  above  group  but  were  not  themselves  rated,  a 
total  of  770  officers. 

The  Identification  listing  of  the  officers  in  these  two  samples 
included  each  officer's  service  number,  basic  career  branch,  completion 
or  not  of  senior  service  college.  This  information  was  obtained  from 
the  Officer  Master  Tape  Record,  as  of  the  end  of  calendar  1970.  Table  1 
lists  the  number  of  subjects  in  the  two  samples,  categorized  by  career 
branch  and  military  education  level. 


PROCEDURE 

Materials  for  completion  and  evaluation  by  the  field  were  sent  by 
mail  to  the  officers  identified  in  the  aforementioned  samples.  (A  sample 
set  of  these  materials  is  attached  at  the  Appendix.)  In  summary,  the 
instructions  to  the  rating  officer  provided  that  he: 

1.  Identify  all  the  officers  on  his  career  branch  roster  about  whom 
he  could  make  a  reasoned  judgment  of  general  officer  potential.  Then, 
from  those  so  identified,  indicate  a  proportion  with  high  general  officer 
potential,  and  a  similar  proportion  with  limited  general  officer  potential. 
Guidance,  based  on  the  size  of  the  career  branch  roster,  w a -3  provided  as 

to  the  number  of  officers  to  be  selected  for  each  category.  For  each 
nomination,  the  rater  then  indicated  his  confidence  in  his  judgment, 
using  a  three-point  scale  of  "much,"  "some,"  "little"  confidence. 

2.  Consider  all  the  officers  on  the  rosters  of  all  career  branches 
except  his  own  who  were  graduates  of  one  of  the  senior  service  colleges. 
Then,  from  those  he  knew,  repeat  the  rating  process  as  accomplished  for 
the  officers  within  his  own  career  branch. 


3*  At  his  option,  submit  explanatory  information  or  comment  about 
his  basis  for  the  selection  of  particular  officers. 

Upon  completion  of  the  foregoing  rating  process,  the  officer  was  to 
indicate  his  opinion  of  the  utility  of  the  nomination  technique  for  deci¬ 
sions  in  promotion,  senior  service  college  selection,  and  assignments. 

He  was  also  invited  to  add  explanatory  comment  to  his  opinion  if  he  so 
wished,  and  finally  to  indicate  his  opinion  about  the  utility  of  the 
rating  procedure  in  the  more  junior  officer  grades  and  the  desirability 
of  a  continuing  program  conducted  on  a  periodic  basis  throughout  the 
officer's  career. 


RESULTS 

The  analyses  reported  here  are  based  on  the  replies  received  from  the 
field  by  2  August  1971,  a  total  of  1775  returns.  These  include:  Sample  1 
for  the  1966  year  group  (subjects  -  respondents)  225  replies,  and  for  the 
1967  year  group  (respondents  only)  314  replies;  Sample  2,  for  the  1969 
year  group  (subjects  -  respondents)  7^8  replies,  and  for  the  1970  year 
group  (respondents  only)  528  replies.  The  distribution  of  respondents 
by  career  branch  and  military  education  level  is  given  in  Table  2. 

The  results  are  presented  and  discussed  in  sections  corresponding  to 
the  separate  objectives:  feasibility,  acceptability,  application,  and 
differentiation. 


FEASIBILITY 

Results  from  the  field  trial  show  that  the  administrative  procedures 
used  here  are  operationally  feasible.  However,  the  results,  including  the 
respondents'  comments,  have  identified  certain  sources  of  potential  error 
and  the  need  for  continuing  administrative  monitoring  in  operational  use 
if  the  program  is  to  be  accepted  and  trusted  by  the  field,  and  if  the 
results  are  to  be  useful  in  personnel  management. 

These  problem  areas  are: 

1.  Identification  of  senior  service  college  cases  was  not  accurate 
in  the  roster  preparation.  Thus,  it  introduced  an  error  bias  into  the 
rating  procedure.  Also,  86  respondents  objected  to  rating  being  made 
only  among  senior  service  college  graduates. 

2.  Response  form  did  not  provide  space  for  rater  to  identify  all 
the  officers  that  he  considered  in  making  his  judgments.  Solution 
requires  an  expanded  form  on  which  the  rater  can  record  all  of  his 
"work-sheet"  processing,  and  a  machine -accomplished,  pre-scoring  screen 
of  completed  forms  to  reject  internally  inconsistent  returns. 
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DISTRIBUTION  UF  RESPONDENTS 
BY  YEAR,  CAREER  BRANCH  &  MILITARY  EDUCATION  LEVEL 
(SAMPLE  1) 

1967 


Total 


23 

12 

38 

1 

20 

47 

2 

95 

13 

6 

19 

17 

8 

13 

314 


Total 


4 

4 

20 

5 

34 

21 

57 

9 

23 

39 

62 

25 

75 

100 

3 

14 

17 

56 

146 

202 

5 

34 

39 

3 

14 

17 

8 

28 

36 

11 

24 

35 

15 

35 

48 

9 

22 

31 

189 

519 

708 

non 

ssc 

ssc 

non 

SSC 

Total 

30 

44 

13 

15 

27 

58 

1 

3 

14 

55 

47 

82 

4 

80 

165 

ie 

ic 

8 

12 

19 

x  — 

17 

pr. 

15 

<— 

17 

2  3 

306 

55° 

Total 

3 •  Need  exists  for  auditing  control  and  for  full  information  to 
the  field  that  auditing  control  is  built  into  the  system  to  insure 
accuracy. 


4.  Some  respondents,  particularly  in  the  small  career  fields, 
were  concerned  that  they  knew  too  few  officers  and  thus  questioned  the 
fairness  and  reliability  of  results  based  on  only  a  few  responses, 
suggesting  that  the  rater  base  be  expanded  to  include  the  immediately 
senior  year  group. 

5.  For  the  senior-year  groups,  for  whom  decision  on  senior  service 
college  and  promotion  to  general  officer  had  already  been  made,  the 
timeliness  of  administration  was  questioned. 


ACCEPTABILITY  OF  THE  PROGRAM 

The  second  aspect  of  this  analysis  deals  with  the  program's  accept¬ 
ability  for  operational  use  in  specific  personnel  management  functions. 

On  the  basis  of  his  reaction  in  the  actual  rating  process,  each 
respondent  was  asked:  "If  an  index  score  (based  on  proportion  of  "high" 
nominations  and  "least"  nominations  for  the  given  ratee)  were  computed 
for  each  officer  in  the  group  you  have  just  considered,  would  you  favor 
it  being  made  available  for  use?"  This  instruction  was  followed  by 
listing  four  specific  personnel  situations,  with  provision  for  choosing 
one  of  the  alternatives — "Yes",  "?",  or  "No" — for  each  situation.  The 
four  questions  and  the  percentage  responding  to  each  alternative  by  year 
group  are  presented  in  Table  3- 

The  responses  indicate  approximately  half  the  respondents  find  use 
of  the  technique  by  promotion  boards  or  senior  service  school  selection 
boards  to  be  acceptable,  while  a  substantial  majority  reject  use  by 
promotion  approving  authority  only  or  use  by  personnel  managers  in 
making  duty  assignments.  The  tabulations  in  Table  3  are  based  on  total 
responses,  and  include  returns  from  respondents  who  expressed  opinions 
on  one  or  more  but  not  all  of  the  four  potential  uses.  However,  Table  3 
does  not  reflect  how  many  individuals  accept  combinations  of  uses,  e.g., 
use  for  both  promotion  and  school  selection.  A  separate  tabulation  was 
therefore  made  for  the  respondents  who  had  expressed  an  opinion  on  use 
in  the  three  situations — promotion  board  considerations,  senior  service 
college  selections,  and  duty  assignments.  (The  alternative,  "use  by 
promotion  approving  authority  only",  was  not  included  in  this  tabulation 
since  its  dependence  on  the  response  given  to  the  "promotion  board 
considerations"  item  led  to  ambiguity  in  interpretation  and  response 
by  these  officers) .  The  results,  showing  the  degree  of  acceptance  for 
each  potential  use  and  each  combination  of  uses,  are  given  in  Table  4. 
These  results  are  shown  for  the  total  respondents  in  each  year  group,  and 
separately  for  senior  service  college  graduates  and  non-senior  service 
college  graduates. 


Table  3 


QUESTIONS  AND  RESPONSES  ON  ACCEPTABILITY  OF  PEER  NOMINATION  SYSTEM 


"1.  Would  you  favor  the  scores  being  made  available  for  use: 
a.  In  Promotion  Board  consideration?" 


it  Yes 

LI 

i  No 

1966 

yr 

gP 

54 

5 

41 

L967 

yr 

gP 

47 

7 

46 

1969 

yr 

gP 

43 

5 

52 

1970 

yr 

gP 

42 

5 

33 

TOTAL  45 

5 

50 

b.  By  Promotion  Approving  Authority  Only? 


it  Yes 

LI 

'jt  No 

1966  yr 

gP 

12 

16 

?2 

1967  yr 

gP 

18 

14 

68 

1969  yr 

gP 

17 

13 

70 

1070  yr 

gP 

19 

16 

69 

TOTAL  17 

14 

69 

c.  In  selection  board  consideration  for  senior  service  college? 


It  Yes 

LI 

it  No 

1Q66  yr 

gP 

56 

5 

39 

1967  yr 

gP 

50 

8 

42 

1969  yr 

gP 

49 

8 

43 

1970  yr 

gP 

42 

7 

51 

TOTAL  48 

7 

45 

d.  By  personnel  manager s  in  making  duty  assignments? 


it  Yes 

LI 

i  No 

1966 

yr 

gp 

35 

7 

58 

1967 

yr 

gp 

28 

12 

60 

1969 

yr 

gp 

30 

9 

61 

1970 

yr 

gp 

26 

9 

69 

TOTAL  29 

9 

62 
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Results  in  Table  4  Indicate  that  for  the  total  group,  a  significant 
minority  favor  use  for  all  three  purposes,  ranging  from  26$  in  the  most 
advanced  group  to  19$  in  the  junior  group.  If  use  for  assignment  were 
eliminated,  those  favoring  use  for  promotion  and  school  selection  would 
add  up  to  46$  (1966),  40$  (1967),  38$  (I969),  and  35$  (1970).  Finally, 
combining  the  results  shown  in  Table  4,  separately  for  senior  service 
college  and  non-senior  service  college  respondents,  the  degree  of 
acceptance  for  each  option  alone  is: 


1966  1967  1969  1970 

yr  gp  yr  gp  _ yr  gp  yr  gp  TOTAL 


Accept  for: 

SSC 

non 

SSC 

SSC 

non 

SSC 

SSC 

non 

SSC 

SSC 

non 

SSC 

SSC 

non 

SSC 

Promotion 

51$ 

56$ 

40$ 

51$ 

39$ 

45$ 

38$ 

41$ 

42$ 

45$ 

Selection  for  SCC 

49$ 

61$ 

46$ 

53$ 

43$ 

51$ 

34$ 

44$ 

44$ 

50$ 

Assignment 

27$ 

41$ 

28$ 

29$ 

30$ 

29$ 

10$ 

28$ 

26$ 

30$ 

These  results  show  a  downward  trend  in  the  acceptability  rates  across 
the  year  groups  from  senior  to  junior,  and  a  significant  difference  between 
senior  service  and  non-senior  service  college  respondents  in  willingness  to 
accept  the  program  for  one  or  more  of  its  potential  uses.  The  calendar 
trend  may  relate  to  a  fear  of  unknown  consequences.  In  the  senior  group, 
with  a  clear  understanding  that  the  program  will  probably  never  involve 
them  operationally,  the  choice  can  be  impersonal;  in  the  junior  group, 
in  the  absence  of  full  information  on  operational  use,  respondents  may 
hesitate  because  of  the  unknown  impact  on  them  personally.  Likewise, 
senior  service  college  graduates  may  be  less  accepting  because  of 
potential  negative  consequence  on  their  present  status. 

Finally,  the  respondents  were  combined  by  career  branch  into  the  group¬ 
ings  proposed  by  the  Officer  Personnel  Management  System  study.  (Because 
of  the  small  number  of  respondents,  present  AG  and  Finance  officers  have 
been  included  in  the  Material  and  Movement  groups).  The  results  for  all 
year  groups  combined,  and  separately  for  senior  service  college  and 
non-senior  service  college  respondents,  are  shown  in  Table  5*  These 
show  two  striking  deviations  from  the  overall  results,  a  markedly  higher 
rate  of  rejection  for  all  purposes  l)  by  senior  service  college  respon¬ 
dents  in  the  Combat  Support  Arms,  and  2)  by  the  non-senior  service 
college  graduates  in  the  Material  and  Movement  grouping.  Except  as 
these  differences  may  reflect  concerns  about  personal  career  opportunity 
between  the  present  and  proposed  system  for  those  two  groups  of  respon¬ 
dents,  there  is  no  evident  explanation  for  the  deviations. 
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Table  5  (cont) 


OFFICERS  RESPONDING  TO  ACCEPTABILITY  QUESTIONS,  BY  CAREER  BRANCH  GROUP 
(Career  branches  grouped  as  proposed  In  Officer  Personnel  Management  System) 


Total 


Favor  use  for  all  three  purposes . 

Favor  use  for  Promotion  and 
Selection  for  SSC. 

Favor  use  for  Promotion  and 
Assignment . 

Favor  use  for  Selection  for  SSC 
and  Assignment. 

Favor  use  for  Promotion  only. 

Favor  use  for  Selection  for  SSC 
only. 

Favor  use  for  Assignment  only. 

Oppose  or  question  use  for  all 
three  purposes. 

TOTAL 


(N-1707) 

Combat  Material  & 
Combat  Support  Movement , 


Arms 

Arms 

Other 

Total 

N 

X 

N 

X 

N 

_% _ 

N 

% 

192 

20 

79 

20 

74 

21 

345 

20 

186 

19 

70 

18 

55 

: 

16 

311 

18 

5 

— 

6 

1 

1 

2 

1  i 

13 

1 

35 

1 

4 

16 

4 

19 

5 

70 

4 

54 

6 

19 

5 

16 

5 

89 

5 

54 

6 

23 

6 

16 

1 

5 

1 

93 

6 

42 

4  ! 

11 

3i 

1 

14 

4  ! 

i 

1  67  4 

397 

41 

171 

43 

151 

43 

l  719  42 

965 

100 

395 

100 

347 

100 

1707 

LOO 

Officer  Personnel  Management  System  Groupings: 

Combat  Arms:  AD,  AR,  FA,  IN 

Combat  Support  Arms:  EB,  CM,  MI,  MP,  SC 

Material  &  Movement,  Other:  OR,  QM,  TC,  AG,  FI 
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The  foregoing  summary  shows  the  results  without  regard  to  estimating 
the  possible  gains  in  acceptability  which  would  accrue  through  correction 
of  those  objectionable  features  consented  on  which  are  amenable  to  change. 
Of  those  officers  who  rejected  all  options  and  who  made  comments,  10 $ 
voiced  objections  to  features  which  could  be  changed.  Some  of  these 
officers  probably  would  come  to  a  degree  of  acceptance  if  changes  were 
made,  but  it  is  impossible  to  estimate  their  number. 

Consents.  Respondents  were  provided  space  and  encouraged  to  comment 
as  they  wished  on  specific  details,  to  elaborate  their  answers,  and  to 
indicate  their  attitudes --either  favorable  or  unfavorable — to  the  general 
concept.  In  the  total  respondent  group,  954  officers  (54$)  provided  a 
total  of  1250  separately  identifiable  consents. 

These  comments  were  analyzed  and  grouped  into  content  categories. 

This  analysis  was  done  in  order  to  identify  the  sources  of  the  officers' 
concern  about  adoption  of  the  program  in  terms  of  accuracy  of  the 
source  data,  apprehension  about  its  operational  application,  or  other 
problem  areas,  as  well  as  to  identify  actions  that  can  be  taken  to  correct 
or  diminish  expressed  negative  reactions  or  to  support  positive  suggestions. 
The  subject  matter  and  frequency  of  mention  of  these  comments  is  presented 
in  Table  6.  In  summary,  the  grouping  and  analysis  of  the  comments 
shows: 


1.  Concern  for  Administrative  Accuracy.  (77  Mentions)^  The  basic 
identifying  data  on  the  ratees — name,  career  branch,  and  completion  of 
senior  service  college--were  obtained  from  the  Officer  Master  Tape  File. 
Six  officers  in  the  1966  sample  and  25  officers  in  the  1969  sample  noted 
that  they  were  in  fact  graduates  of  a  senior  service  college  but  were 
not  so  Identified,  while  three  officers  in  the  1969  sample  conversely 
noted  that  they  were  not  graduates  but  were  so  identified.  One  officer 
in  the  1969  sample  noted  that  he  was  listed  in  the  wrong  career  branch. 
(When  checked,  a  significant  proportion  of  these  errors  were  also  found 
duplicated  in  the  Army  Register,  January  1971- )  Two  officers  listed  in 
the  1969  sample  were  already  serving  as  general  officers,  while  some 
twenty  had  died  or  retired  in  the  period  intervening  between  preparation 
and  publication  of  the  rosters. 

In  addition  to  noting  this  type  of  error,  the  officers  expressed 
concern  for  its  possible  unfair  Impact  in  an  operational  situation. 
Nineteen  officers  suggested  that  the  program  should  not  be  adopted 
unless  a  continuous  monitoring  system  to  insure  complete  accuracy  were 
developed. 


1, 

Some  officers  mentioned  more  than  one  concern 
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Table  6 


ACCEPTABILITY  OF  PEER  RATING  PROGRAM 

Tabulation  of  Conmenta 
(1250  comments  by  954  Respondents) 


Number  of 


Administrative  Accuracy  Mentions 

Errors  in  identification  of  SSC  subjects  30 

Inclusion  of  retired  or  deceased  16 

Some  subjects  already  promoted  to  0-7  2 

Instructions  should  be  expanded  2 

Lacks  control-develop  administrative  monitoring  19 

Administrative  burden  _8 

77 

Sample  Too  Limited 


Don't  limit  cross-branch  to  senior  service  college 

graduates  85 

Expand  more  than  one  fiscal  year  on  subject  roster  26 

Should  include  seniors  and  contemporaries  as 

subjects  8 

Expand  definition  of  contemporaries  6 

125 

Control  of  Results 

For  officers'  TAG  and  career  branch  file  only  5 

Only  for  promotion  board,  SSC  selection  board,  and 
approving  authority  3 

To  the  subject  officer  only  14 

22 

Limit  Application 

Use  only  with  other  selection  procedures  53 

Use  for  self-evaluation  only  9 

Use  only  as  "tie-breaker"  or  other  information 

not  available  11 

73 

Change  Rating  Techniques 

Use  only  "High"  ratings  9 

Use  only  ratings  given  with  "Much"  confidence  4 

Restrict  process  to  mm  career  branch  only  6 

Restrict  process  to  outside  own  career  branch  only  1 

Low  ratings  should  be  justified  by  factual  comment  12 

Respondent  should  not  be  restricted  to  specific  numbers 

to  be  rated  29 

Need  more  complete  identification  of  subjects  15 

Expand  to  ratings  of  junior  year  group  by  seniors  16 

Use  only  with  officer  serving  (or  have  served) 

together  10 

Expand  to  ratings  on  specific  performance  characteristics  13 
Use  a  check  on  GER  (OEl)  and  weight  less  than  OER  8 

Ratings  on  juniors  OK,  contemporaries  No,  seniors  No.  13 


Table  6  (coot) 

Comments  about  an  officer  should  be  validated  by  several 
mentions  before  acceptance 

Rating  should  not  be  used  until  accumulated  over  several 
cycles 

"High"  and  "Low"  only  unfair,  add  "Medium" 

Define' promotion  potential* 

Use  a  rank  order  scheme 

Compensate  for  "High"  and  "Low"  rater  bias 

Use  at  graduation  from  SSC,  or  not  before  15th  year 

Positive  Comments 

Good  addition  for  0-6  and  0-7  level 
Good  in  0-4  to  0-6  level 

Picks  up  officer  qualities  not  observed  by  seniors 
Useful  in  lower  grades 
Will  accept  if  official  system 
"Great  technique" 

General  favorable  endorsement 


Negative  Comments 

Favors  better  known  officers 
Lead  to  popularity  contest 
Too  subjective 

Ratings  based  on  friendship,  compatability,  rivalry 
factors 

Will  add  to  present  system  of  "Cliques" 

Competition  and  jealousy  will  ruin  it 
Officers  won't  have  equal  opportunity  for  exposure 
Weakens  chain  of  command-must  supervise  officer  to 
rate  him 

"Worthless-comp letely  disagree." 

Destroy  trust  among  officers 

Create  conformity-destroy  creativity 

Not  valid  - 

Danger  of  collusion 

Dangerous-if  subject  learns  his  score 

Army  too  large  for  this  system 

Peers  lack  impartiality 

Too  impersonal 

Won't  compare  favorably  with  promotion  board  system 
Can't  be  uniformly  applied  to  all  officers 
Reduce  officers  to  publicity  seekers 
Great  pressure  on  rater  by  other  officers 
"Hearsay"  and  lack  of  control 
General  negative  comment 


2 


3 

8 


1 

2 

1 


7 

6 

12 

2 


9 

11 

118 


33 

73 

50 

70 

12 

11 

12 

10 

24 

8 

4 

22 

4 

5 

11 

4 

4 

4 

4 

3 

2 

9 

-31 

416 


Miscellaneous 


Don't  know  enough  officers  to  rate  196 

Not  needed-present  system  adequate  32 

Forced  to  put  good  officers  in  "Low"  category  5 

Work  on  improving  OER  instead  4 

Withhold  decision  until  results  are  evaluated  10 

Beware  "  Unknown"  officer  not  penalized  10 

Good  if  handled  carefully  and  objectively  _ £ 

261 
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2.  Sample  too  limited  (125  Mentions).  The  predominance  of  comments 
here  (85)  Indicated  objection,  and  some  resentment,  to  restriction  of 
choice  to  senior  service  college  graduates  outside  one's  own  career 
branch.  Other  comments  questioned  the  definition  of  associate  (year 
of  promotion  to  colonel  should  not  decide).  Finally,  suggestions  were 
offered  to  expand  the  rater  group — include  more  than  one  year  group, 
include  both  seniors  and  contemporaries. 

5.  Control  of  results  (22  Mentions).  These  comments  concern  access 
to  the  rating  information.  One  group  of  respondents  proposed  restriction 
to  use  for  senior  service  college  selection  and  promotion  purposes  only, 
other  respondents  favored  release  to  the  officer  himself  for  his  guidance, 
and  others  suggested  inclusion  in  the  personnel  file. 

4.  Limit  Application  (73  Mentions).  The  comments  in  this  area 
generally  indicate  a  misapprehension  of  the  possible  application  of  the 
program,  viz.,  53  commented  that  it  should  be  used  only  in  connection 
with  other  selection  procedures,  11  said  only  if  other  materials  were 
not  available,  and  9  suggested  use  for  self-evaluation  only. 

5.  Suggestions  for  Changes  in  the  Technique  (158  Mentions).  These 
comments  refer  to  specific  aspects  of  the  present  field  administration 
and  proposed  modifications  in  various  details  of  the  rating  procedure. 

The  comments  are  grouped  into  19  topics  of  suggested  change  or  modifica¬ 
tion  which  will  be  evaluated  and  incorporated  where  appropriate  in 
subsequent  revisions  of  the  procedures. 

6.  Positive  comments,  non-specific  (ll8  Mentions).  The  comments 
grouped  here  are  those  in  which  the  respondent  provided  verbal  reinforce¬ 
ment  to  the  decisions  he  recorded  on  the  acceptability  items,  but  which 
do  not  relate  to  specific  aspects  of  the  procedures. 

7.  Negative  Comments,  non-specific  (416  Mentions).  In  these  comments 
the  respondent  expressed  real  or  imagined  concerns  as  his  reasons  for 
opposition  to  adoption  of  the  program  without  regard  to  specific  aspects. 

8.  Miscellaneous  (261  Mentions).  These  conments  reflect  the  caution 
and  concern  of  the  respondent  in  qualifying  his  decisions  on  acceptability 
of  the  procedures.  The  major  concern  in  this  area  (196  Mentions)  is 

that  the  respondent  did  not  knew  enough  officers  well  enough  to  rate 
them  and  thus  was  dubious  of  the  validity  of  the  procedure. 

No  specific  action  in  response  to  the  negative  and  miscellaneous 
comments  can  be  proposed,  except  that  fuller  information  and  explicit 
explanation  responsive  to  the  areas  of  concern  may  alter  the  attitudes 
of  some  of  these  officers. 
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Many  of  the  concerns  expressed,  as  tabulated  above,  basically  reflect 
a  lack  of  Information  on  the  utilization  of  results  and  an  unwillingness 
to  fully  accept  the  procedure  until  they  understand  Its  utility  and  the 
limits  placed  on  the  availability  of  the  results.  These  concerns  dictate 
a  need  for  expanded  lnformatlon/explanatory  content  In  the  instructional 
materials  provided  for  field  completion. 

Use  of  the  Program  In  the  Lower  Officer  Grades.  In  addition  to 
considering  use  of  this  rating  system  in  the  grade  of  colonel,  as 
discussed  above,  the  officers  were  asked  to  consider  desirability  of  its 
use  in  the  lower  grades.  (The  responses  reported  here  are  combined  for 
the  four  separate  year  groups,  since  there  are  no  real  differences  be¬ 
tween  the  groups  in  the  opinions  expressed.)  The  results  for  the  partic¬ 
ular  questions  are: 

"2.  Would  you  favor  operational  use  of  such  a  program  in  the  lower 
officer  grades?" 


(1716  responses)  40$  yes 
4$  ? 

56$  no 

"2. a  If  so,  at  what  grades?  (More  than  one  may  be  marked.)" 

(1240  responses  from  687  officers) 

4$  1LT 
9$  CPT 
57$  MAJ 
50$  LTC 

"3.  Would  you  favor  such  a  program  at  regular  intervals  in  each 
officer's  career?" 

(1671  responses)  44$  yes 
4$  ? 

52$  no 
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”3. a  If  so,  at  what  Intervals  ...  beginning  at  which  year  of  commis¬ 
sioned  service?"  (6Q5  responses) 

Interval 


2  yrs 

3  yrs 

4  yrs 

Cumulative1 

Beginning  at 

2d  yr 

5* 

7 * 

11* 

23* 

6th  yr 

3* 

17* 

21* 

64* 

10th  yr 

2$ 

15* 

19* 

100$ 

Cumulative* 

10$ 

49* 

100$ 

In  the  interpretation  of  this  table,  a  logical  progression  of  accep¬ 
tance  is  assumed,  i.e.,  those  accepting  a  two-year  interval  would 
also  accept  the  three  or  four  year  interval,  those  accepting  "begin 
at  2d  year"  would  also  accept  at  6th  and  10th  year. 


Results  on  usage  in  the  lower  grades  were  not  a  focal  point  of  concern 
to  the  responding  officers  and  thus  drew  very  limited  comment.  The  few 
comments  suggested  that  a  rating  program  would  be  useful  in  the  career 
course  or  Command  and  General  Staff  College,  but  unmanageable  in  other 
situations.  No  suggestions  were  made  concerning  applicability  of  results 
of  such  a  program. 


APPLICATION 

Results  presented  in  Tables  3>  4,  and  5>  and  the  analysis  of  comments 
reported  above,  indicate  clearly  that  the  most  acceptable  applications  of 
the  technique  are  for  senior  service  college  selection  or  promotion  board 
action  and  next  most  acceptable  for  the  combination  of  the  two.  If  the 
technique  were  instituted  for  senior  service  college  selection,  it  would 
have  to  be  applied  to  a  group  newly  or  recently  designated  for  promotion 
to  colonel.  Application  at  this  point  would  eliminate  the  designation  of 
senior  service  college  graduate,  placing  all  subjects  in  the  same  compet¬ 
itive  situation,  and  so  eliminate  a  source  of  the  most  serious  "bug"  in 
the  experimental  tryout  and  of  many  negative  comments  on  the  system. 

Insofar  as  completion  of  senior  service  college  is  a  favorable  consider¬ 
ation  by  boards  for  promotion  to  general,  the  technique  would  Influence 
the  promotion  process  without  being  a  direct  determii  of  that  process. 
Initial  use  for  senior  service  college  selection  might  also  lead  to  greater 
acceptability  of  use  for  promotion  board  action. 

There  appears  to  be  little  question  that  use  for  duty  assignment  is 
not  acceptable.  Likewise,  use  at  times  and  grades  below  lieutenant  colonel 
is  favored  by  only  a  minority  of  the  respondent  colonels. 
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ADEQUACY  OF  DIFFERENTIATION 

The  results  reported  here  are  compiled  separately  on  the  1966  colonel 
year  group,  whi-h  Includes  nominations  made  by  their  awn  group  and 
nominations  of  their  members  made  by  the  1967  colonel  year  group  (Sample  l) 
and  for  the  1969  colonel  year  group,  which  includes  nominations  made  by 
their  own  group  and  nominations  of  their  members  made  by  the  1970  colonel 
year  group  (Sample  2). 

First  tabulations  were  made  of  the  number  of  times  each  subject  officer 
was  "known  well  enough  to  rate"  by  respondents  in  his  own  career  branch. 
Table  7  summarized  results  of  these  tabulations,  showing  the  number  of 
subject  officers  identified  a  given  interval  of  times  by  the  peer  respon¬ 
dent  group  and  the  junior  respondent  group.  Table  8,  from  the  same 
tabulations,  shows  the  number  of  subject  officers  identified  by  a  given 
percentage  of  the  respondents. 

Tabulations  similarly  were  made  for  the  number  of  times  each  senior 
service  college  subject  was  "known  well  enough  to  rate"  by  respondents 
outside  of  his  career  branch.  Table  9  summarizes  the  results  of  these 
tabulations  for  the  subjects  in  each  career  branch,  showing  the  number 
of  officers  identified  a  given  interval  of  times  by  the  peer  respondent 
group  and  the  junior  respondent  group.  Table  10,  from  the  same  tabulations 
shows  the  number  of  subject  officers  identified  by  given  percentages  of 
respondents. 

Results  shown  in  Tables  7  through  10  are  underestimated,  in  that  a 
sizable  proportion  of  respondents  did  not  identify  the  officers  that  they 
considered.  However,  the  results  show  that  when  peers  and  juniors  are 
considered  together  as  the  rating  group,  the  composite  of  raters  probably 
identify  an  adequate  base  of  known  officers  for  reliable  judgment,  except 
possibly  in  the  small  branches.  To  increase  the  stability  and  reliability 
of  the  composite  ratings  (and  to  allay  significant  concern  on  the  part  of 
the  participants  as  noted  in  their  comments),  consideration  should  be 
given  to  expanding  the  rater  base  through  inclusion  of  the  year  group 
immediately  senior  to  the  rated  group  and/or  expanded  use  of  cross-career 
branch  ratings. 

Tabulations  were  also  made  of  the  number  of  subject  officers  each 
respondent  considered  that  he  "knew  well  enough"  to  rate  in  his  own  career 
branch.  Table  11  summarizes  the  results  from  these  tabulations,  showing 
the  number  of  subject  officers  considered  by  peer  and  junior  respondents. 
Table  12  from  the  same  tabulations  shows  the  number  wf  respondents  who 
identified  a  given  percentage  of  the  subject  officers. 

Tabulations  were  made  similarly  for  the  number  of  senior  service  col¬ 
lege  subject  officers  that  each  respondent  considered  that  he  "knew  well 
enough  to  rate"  outside  of  his  own  career  branch.  Table  13  shows  these 
results  for  the  peer  and  junior  respondents  in  each  career  branch,  and 
Table  14  shows  the  percentages  of  subjects  identified  by  the  shown  number 
of  respondents. 
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DISTRIBUTION  OF  PERCENTAGE  OF  TIMES  SUBJECT  RECOGNIZED  BY  RESPONDENT  WITHIN  CAREER  BRANCH 


TIMES  SUBJECT  RECOGNIZED  BY  RESPONDENT  WITHIN  CAREER  BRANCH 


F  SUBJECTS  RECOGNIZED  BY  RESPONDENTS  WITHIN  CAREER  BRANCH 


Table  13 


SENIOR  SERVICE  COLLEGE  SUBJECTS  RECOGNIZED  BY  Tn£  RESPONDENT 
(by  Career  Branch  of  Respondent) 


The  results  given  in  Tables  11  through  14  show  that  generally  respond¬ 
ents  individually  identify  an  adequate  number  of  subject  officers  to  form 
a  sound  basis  for  nomination.  From  Table  12  it  can  be  noted  that  in  the 
smaller  career  branches  with  few  subject  officers,  the  individual  respond¬ 
ents  generally  recognize  a  quite  high  percentage  of  the  subjects. 

Similarly  the  results  in  Table  13  and  14  show  that  the  respondents 
identify  an  adequate  number  of  subject  officers  to  form  a  sound  basis  for 
nomination.  The  lower  percentages  of  recognitions  in  Table  14  as  compared 
with  Table  12  reflect  the  large  number  of  subject  officers  outside  the 
respondents'  career  branch  rather  than  a  lack  of  adequate  numbers  identified 
by  the  respondents. 

For  the  ratings  completed  on  subject  officers  by  respondents  within 
their  own  career  branch,  the  results  were  compiled  into  two  separate  scores 
for  each  rated  officer.  The  first  considered  nominations  only  without 
regard  to  the  degree  of  confidence  indicated  by  the  respondent  in  making 
the  nomination,  and  the  second  weighted  each  nomination  by  its  confidence 
level  C  (much  *  3>  some  =  2,  or  little  ■  1). 

Further,  since  many  respondents  did  not  nominate  equal  numbers  of 
subjects  in  the  "High"  and  "Low"  categories,  or  in  the  extreme  case  nomi¬ 
nated  "High"  subjects  only,  a  factor  was  included  in  the  scoring  equation 
to  correct  for  the  difference  between  total  number  of  "High"  and  total 
number  of  "Low"  nominations.  Finally,  a  correction  was  included  for  the 
different  number  of  subjects  and  respondents  between  the  separate  career 
branches,  to  equate  the  score  ranges  across  the  various  career  branches. 

The  scoring  equations  are: 


Unweighted  Score  #1 


Weighted  Score  #2 


(R  -  <nh 
(»  •  <"» 


EL  -  EH 

nR  -  (EL  +  EH) 

EL  -  EH 
nR  EL  +  EhT 


)] 

•)] 


X  50 

X  50 


where : 

N  and  N  are  the  number  of  high  and  low  nominations  for  the 
H  L 

individual  subject; 

R  is  the  number  of  respondents; 

EH  and  EL  are  the  total  number  of  high  and  low  nominations  for 
the  subject  group; 


EC„  and  ZCT  are  the  sums  of  the  weighted  high  and  low  nominations 
for  the  individual  subject; 

n  is  the  number  of  subject  officers  in  the  group. 

Table  15  shows  the  mean  scores  and  the  standard  deviation  of  these 
scores  for  the  subjects  in  each  career  branch  separately  for  the  two 
samples.  It  also  shows  the  reliability  estimates  of  the  scores  for  each 
career  branflh--the  degree  of  agreement  (or  stability)  of  scores  from 
nominations  made  by  the  odd-numbered  respondents  and  the  nominations  made 
by  the  even-numbered  respondents,  and  the  estimated  (corrected)  reliability 
for  the  total  respondent  group.  Finally,  it  shows  the  correlation--the 
relative  agreement — between  score  #1  and  score  #2  for  the  subjects  in 
the  total  rated  group. 

In  this  table  it  will  be  noted  that  the  mean  scores  for  score  #1 
approximate  zero,  as  defined  by  the  scoring  equation,  and  the  correlations 
between  score  #1  and  score  #2  are  extremely  high  (.97  or  above),  which 
show  that  there  is  no  added  information,  and  therefore  no  advantage  to 
the  inclusion  of  the  level  of  confidence  in  the  scoring  process. 

The  standard  deviations  which  indicate  the  score  dispersions  of  the 
individual  subjects  and  the  degree  to  which  these  individuals  can  be 
confidently  rank-ordered  are  sufficiently  large  to  provide  adequate 
differentiation  among  the  subjects.  The  reliabilities  are  sufficiently 
high  to  indicate  acceptable  stability  of  the  rating  scores. 

In  the  cases  of  the  numerically  small  career  branches,  with  few  subjects 
and  respondents  (as  shown  in  Tables  1  and  2)  these  results  should  be  inter¬ 
preted  with  caution.  For  sample  1,  these  career  branches  are  AG,  CM,  FI, 
and  MP;  for  sample  2,  CM  and  FI. 

For  the  ratings  completed  on  senior  service  college  subjects,  the  two 
scores  as  defined  above  were  also  computed  from  the  nominations  made  by 
respondents  from  all  career  branches  except  their  own.  Table  16  shows 
these  results  in  the  same  manner  as  provided  in  Table  15  for  withln-career 
branch  results. 

For  the  results  shown  in  this  table,  it  should  be  noted  that  the  mean 
for  score  #1  is  constrained  by  the  scoring  equation  to  approximate  zero 
only  for  the  total  of  all  career  branches,  since  the  subjects  were  consid¬ 
ered  as  a  single  group  without  regard  to  career  branch.  Thus,  differences 
in  career  branch  mean  scores  reflect  quality  differences  as  viewed  by  the 
respondents.  In  the  results  shown  in  Table  16,  there  is  no  evidence  of 
significant  variation  between  the  branches,  or  branch  bias  on  the  part  of 
the  respondents. 

As  with  the  results  for  the  withln-career  branch  ratings,  the  mean, 
standard  deviation,  and  reliability  for  *he  total  group  provide  a  sound 
basis  for  rank-ordering  the  subjects  with  adequate  dif ferentation. 
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Another  question  investigated  concerned  the  military  education  level 
(SSC  or  non-SSC)  and  the  seniority  (peer  or  juniors)  in  relationship  to 
the  number  of  subject  officers  within  their  own  career  branch  that  they 
identified  as  knowing  well  enough  to  rate.  For  this  analysis  an  "associa¬ 
tion  score"  was  computed  for  each  respondent,  as  follows: 

Association  score  *  £  ( H  +  L)  X  100 

S 

where  H  and  L  are  the  number  of  high  and  low  nominations  made 
by  the  respondent,  and  S  is  the  number  of  subjects  in  the 
career  branch. 

Table  17  summarizes  the  means  and  F-values  of  this  analysis  by  the 
education  level  of  the  respondents,  while  Table  19  shows  the  results  by 
the  peer-junior  category  of  respondents  for  those  career  branches  having 
an  adequate  number  of  respondents.  In  general,  the  results  show  SSC 
graduates  to  be  better  acquainted  and  able  to  nominate  more  subjects 
than  non-graduates,  but  no  significant  differences  were  found  between 
peer  and  junior  respondent  groups. 

The  cross-career  branch  relationships  between  respondents  and  subjects 
were  also  studied  to  determine  acquaintanceship  affinity  between  career 
branches  as  these  are  to  be  regrouped  according  to  the  concepts  of  the 
Officer  Personnel  Management  System.  For  this  purpose,  senior  service 
college  graduate  subjects  were  grouped  into  the  Combat  areas  (AD,  FA,  AR, 
IN),  Combat  Support  area.  Material  and  Movements  area,  and  "other"  area, 
and  the  association  scores  computed  as  described  above.  (Because  of  its 
size.  Infantry  branch  was  divided  into  random  halves,  each  treated  as  a 
separate  career  grouping.  Branches  with  inadequate  numbers  of  respondents 
were  dropped  from  the  analysis) .  Table  21  summarizes  the  means  and 
F-values  for  these  analyses  of  variance. 

The  results  show  generally  significant  relationships  between  OPMS-type 
groupings  and  the  acquaintanceship  span  of  the  respondents.  Indicating 
that  the  rating  methodology  investigated  in  this  study  would  appropriately 
apply  to  the  new  career  groupings.  Results  for  Quartermaster  respondents 
show  significant  interactions  with  military  education  level  while  the 
results  for  Field  Artillery  respondents  show  significant  interactions  with 
peer- junior  status.  These  results  are  presented  in  Tables  18  and  20. 

Finally,  the  data  were  examined  to  determine  if  there  were  significant 
quality  variations  between  branches  of  the  subjects,  as  perceived  by  the 
respondents,  l.e.  were  "High"  nominations  disproportionately  distributed 
across  the  separate  career  branches?  The  findings  indicated  no  tendency 
for  respondents  to  favor  one  branch  at  the  expense  of  another,  even  though 
they  were  Influenced  by  acquaintanceship  factors  in  determining  the  branches 
in  which  nominations  (both  "High"  and  "Low")  were  concentrated. 
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Table  17 


MEANS  AND  F-VALUES  OF  ASSOCIATION  SCORES  BT 
MILITARY  EDUCATION  LEVEL  OF  RESPONDENTS 


Military  Education  Level 

Branch 

SSC 

Non-SSC 

F-Value 

AD 

9.83 

6.00 

6.773** 

FA 

8.79 

5.75 

15.149** 

AR 

9.86 

6.03 

12.026** 

IN 

10.15 

7.41 

9.018** 

EN 

6.33 

4.82 

1.962 

OR 

5.81 

4.00 

2.369 

SC 

6.94 

3.42 

6.042** 

MP 

6.58 

3.79 

less  than  1 

QM® 

7.42 

3.51 

7.154** 

TC 

4.97 

3.48 

leaa  than  1 

**  p<.01 

a  Significant  interaction  with  branch  (see  Table  18) . 


Table  18 

MEANS  AND  F-VALUES  OF  ASSOCIATION  SCORES  FOR  QM  BRANCH 
(BROKEN  DOWN  BECAUSE  OF  INTERACTION) 

(Branches  were  grouped  in  conformity  with  the  objectives  of  OPMS: 
Cj-AD+FA,  C2-AR,  C3-IN/2,  C4-IN/2,  C5-Combat  Support  Arms,  and 
Cg-Materials  and  Movement,  others) 


Respondents 

Branch  Group 

(Subjects) 

(Military 

School) 

Cl 

C2 

C3 

C4 

C5 

c6 

F-Value 

SSC 

Non-SSC 

3.38 

2.88 

6.50 

2.71 

6.63 

2.65 

2.50 

2.47 

6.38 

2.94 

19.13 

7.41 

11.5089** 

1.1727 

F-Value 

.0195 

1.1221 

1.2519 

.0001 

.9244 

10.7305** 

Table  19 


MEANS  AND  F-VALUES  OF  ASSOCIATION  SCORES  BY 
PEER-JUNIOR  RELATIONSHIP  OF  RESPONDENTS 


Branch 

1966 

Year  Group 

1967 

F-Value 

AD 

7.17 

7.28 

less  than  1 

FA* 

6.26 

7.64 

3.092 

AR 

8.77 

7.72 

less  than  1 

IN 

9.69 

8.19 

1.118 

EN 

5.15 

6.17 

2.282 

OR 

4.63 

4.75 

less  than  1 

a  Significant  Interaction  with  branch  (see  Table  20). 


Table  20 

MEANS  AND  F-VALUES  OF  ASSOCIATION  SCORES  FOR  FA  BRANCH 
(BROKEN  DOWN  BECAUSE  OF  INTERACTION) 

(See  Table  18  headnote) 


Branch  Group  (Subjects) 


Respondents 

C2 

C3 

C4 

C5 

C6 

F-Value 

Peer 

8.71 

8.29 

6.26 

4.71 

3.34 

3.3873* 

Junior 

7.66 

7.57 

7.17 

6.60 

9.19 

.6032 

F-Value 

.1634 

.0743 

.1225 

.5199 

5.9061** 

**  p<.01 

*  p<.05 
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Table  21 


MEANS  AND  F-VALUES  OF  ASSOCIATION  SCORES  BY  BRANCH  GROUP 
OF  SUBJECTS  FOR  EACH  BRANCH  OF  RESPONDENTS 
(See  Table  18  headnote) 


Branch 


Branch  (Subjects) 


(Respondents) 

C1 

C2 

C3 

C4 

C5 

C6 

F-Value 

AD 

15,88® 

4.93 

5.66 

5.02 

4.91b 

6.91 

14.106** 

FAC 

8.11 

7.88 

6.78 

5.79u 

6.70 

3.198 

AR 

11.91® 

8.17 

9.28 

4.72b 

6.31 

7.930** 

IN 

9.76 

10.12* 

7.00b 

8.42 

7.823** 

EN 

6.83 

5.54 

5.46 

6.54. 

3.60 

6.40 

1.440 

OR 

3.39 

2.81 

5.00 

2.13b 

4.61 

10.26® 

5.130** 

SC 

4.04 

2.78 

4.43 

3.57 

4.65 

8.39 

2.932 

MP 

4.50 

5.58 

4.75 

4.17K 

1.67b 

7.67® 

2.972* 

QM“ 

3.04 

3.92 

3.92 

2.48b 

4.04 

11.16® 

9.858** 

TC 

2.13 

3.96 

3.96 

2.91 

4.17 

6.09 

1.294 

*  Highest  mean  In  row  with  significant  F-values. 
b  Lowest  mean  In  row  with  significant  F-values. 

c  Significant  Interaction  with  year  group  (see  tables  19  and  20) . 
d  Significant  Interaction  with  military  education  level  (see  tables  17 
and  18  )  • 

**  p<.01 

*  p<.05 
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SUMMARY  AND  CONCLUSIONS 


In  response  to  a  DCSPER  request,  an  experimental  tryout  of  a  peer 
nomination  system  among  Army  colonels  for  possible  use  in  promotions, 
senior  service  college  selection,  and  assignments  was  conducted  by  the 
Army  Research  Institute.  Rosters  of  the  group  designated  for  promotion 
to  colonel  in  1966  were  distributed  to  all  colonels  in  that  group  and  in 
the  1967  promotion  group  as  well.  Similarly,  rosters  of  the  group 
designated  for  promotion  in  1969  were  distributed  to  that  group  and  to 
those  designated  for  promotion  in  1970.  Each  respondent  was  requested 
to  nominate  a  given  number  of  officers  in  his  career  branch  as  "high  in 
general  officer  potential"  and  an  equal  number  as  "low."  In  addition, 
he  was  requested  to  repeat  this  nomination  procedure  for  officers  outside 
of  his  own  career  branch,  who  were  also  senior  service  college  graduates. 
Finally,  he  was  asked  to  consider  the  proposed  system  for  operational 
use,  and  to  indicate  his  attitude  toward  its  use  for  each  of  the  three 
purposes--promotion,  school  selection,  and  assignment. 

Results  from  nominations  on  the  1966  and  1969  groups  and  attitude 
responses  of  the  four  respondent  year  groups  were  as  follows: 


FEASIBILITY 

A  number  of  administrative  problems  were  identified,  such  as  inaccura¬ 
cies  in  the  officer  master  tape  output,  currency  of  organizational  address, 
and  minor  misunderstanding  of  questions.  These  appeared  to  be  readily 
solvable.  There  wae  some  evidence  that  an  expanded  respondent  group  might 
be  needed  for  a  given  career  roster,  especially  in  the  smaller  career 
branches. 


ACCEPTABILITY 

Overall  acceptability  was  greatest  for  the  most  senior  group  (1966) 
and  least  for  the  most  Junior  (1970).  Likewise,  the  overall  acceptability 
was  higher  for  non-senior  service  college  respondents  than  for  senior 
service  college  respondents.  These  findings  may  be  interpreted  as  Indicating 
greater  concern  for  the  effect  of  such  a  system  on  their  own  careers  by 
the  more  junior  respondents  and  by  senior  service  college  graduates,  since 
many  of  the  more  senior  respondents  are  beyond  the  decision  point  of 
promotion  to  general  or  selection  for  senior  service  college.  When 
respondents'  career  branches  are  grouped  as  proposed  by  the  Officer 
Personnel  Management  System,  no  significant  differences  in  acceptability 
are  found  among  these  branch  groups.  The  overall  percentage  accepting 
the  proposed  technique  was  20$  for  all  three  purposes,  44$  for  promotion 
alone  or  in  combination,  48$  for  senior  service  college  selection  alone 
or  in  combination,  and  29$  for  assignment  alone  or  in  combination. 


More  than  half  the  respondents  added  free  comment  to  their  attitude 
responses,  with  concern  for  expanding  the  rater  group,  for  accuracy  of 
data  on  rosters  (and  on  the  officer  master  tape),  and  for  assuring  that 
the  technique  be  used  only  In  conjunction  with  other  indicators  among  the 
consents  to  which  the  technique  could  be  made  responsive.  Among  the 
negative  comments  not  resolvable  the  concern  that  the  technique  would  be 
a  popularity  contest  and  encourage  distrust  among  officers  was  frequently 
mentioned. 

A  majority  (60$)  of  the  respondents  oppose  or  question  use  of  the 
technique  in  the  lower  grades. 


APPLICATION 

The  most  acceptable  point  of  application  appeared  to  be  among  newly 
designated  colonels  to  permit  use  for  senior  service  college  selection 
(thus  avoiding  the  military-education- level  dichotomy  among  the  subject 
officers)  and  possibly  promotion  later.  Expansion  of  the  respondent 
group,  to  include  combined  career  branches  and/or  a  group  one  year  senior 
to  the  ratee  group,  also  was  indicated. 


ADEQUACY  OF  DIFFERENTIATION 

The  distribution  of  subject  nomination  scores  demonstrated  that  the 
technique  yields  adequate  differentiation  and  reliability  for  operational 
use,  if  the  question  of  acceptability  is  resolved.  The  results  showed  that 
nominations  weighted  by  the  expressed  confidence  of  the  respondent  do  not 
improve  differentiation  beyond  that  obtained  from  the  unweighted  nominations. 
Validity  of  the  technique  will  be  studied  at  a  later  period  when  an  appro¬ 
priate  criterion  of  promotion  or  selection  for  senior  service  college  can  be 
established  for  the  subject  officers. 

In  conclusion,  it  should  be  noted  that  the  encouraging  results  of  this 
study  were  obtained  under  experimental  conditions  in  which  all  participants 
were  aware  of  its  confidential  research  nature  and  protection  from  adminis¬ 
trative  review;  thus,  the  quality  of  the  results  obtained  may  not  necessarily 
generalize  to  an  operational  situation.  Also,  some  of  the  reliability  and 
acceptability  obtained  in  this  study  is  substantiated  by  very  frank 
statements  about  specific  subject  officers.  Thus,  it  should  be  recognized 
that  if  such  comments  are  solicited  for  operational  purposes,  the  specific 
individuals  having  access  to  the  comments  must  be  limited  and  identified. 


.wi 
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DEPARTMENT  OF  THE  ARMY 

OFFICE  OF  THE  ADJUTANT  GENERAL 
WASHINGTON.  O.C.  XOSIO 


AGAO-KC  (1  Mar  71)  DCSPER-CSD 

SUBJECT:  Associate  Ratings  of  Senior  Officer  Potential 


TO: 


1.  Department  of  the  Army  is  conducting  a  continuing  research  program 
to  improve  methods  for  evaluation  of  officer  potential.  Currently,  a 
portion  of  this  research  concerns  investigation  of  the  acceptability 
and  validity  of  evaluations  made  by  colonels  of  their  contemporaries. 

As  a  phase  of  this  research,  field  evaluations  are  being  requested  at 
this  time  from  those  officers  selected  for  promotion  to  the  grade  of 
colonel  in  years  FY-66,  67,  69  and  70. 

2.  Inclosed  are  the  instructions,  rating  forms  and  opinion  question¬ 
naire  for  your  completion  in  this  field  evaluation.  Results  from  the 
field  are  for  research  purposes  only  and  will  be  held  in  complete  con¬ 
fidence.  No  data  identifying  individual  raters  or  ratees  will  be  made 
available  for  operational  review.  It  is  stressed,  however,  that  results 
obtained  in  this  research  will  be  instrumental  in  deciding  f ossible  use 
of  such  a  rating  program  operationally  in  subsequent  years.  Therefore, 
it  is  requested  that  you  complete  all  evaluations  aB  you  would  if  the 
results  were  to  be  used  as  a  basis  for  operation! I  decisions,  and  that 
you  respond  to  the  opinion  questionnaire  fully  aid  candidly. 

3.  This  is  an  exempt  report 
AR  335-15. 

BY  ORDER  OF  THE  SECRETARY  OF 


loci 

as 


Major  General,  USA 
Acting  The  Adjutant  General 
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MANUAL  FOR  ACCOMPLISHING  THE 
ASSOCIATE  RATING  OF  SENIOR  OFFICER  POTENTIAL 

FORM  1 


Not  to  ka  Mown  to  uMuthorind  parsons 
Mat  to  ka  raproducad  in  any  torn 
aritoaat  toa  apaelfie  panniaalon  of  toa 

DIRECTOR.  BEHAVIOR  AND  SYSTEMS  RESEARCH  LABORATORY 
ofnci  or  iMf  cmv  or  mmarch  and  ocvclotmcmt 

I  D^ARTMtNT  OT  TH*  ARMY 


PT  4789 
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ACCOMPLISHING  THE  ASSOCIATE  RATING  OF 
SENIOR  OFFICER  POTENTIAL,  FORM  1 


1.  General  Instructions 

a.  The  procedures  outlined  here  for  the  collection  of  field  data 
are  part  of  an  Army  research  program  to  develop  new  techniques  and 
sources  of  Information  on  the  performance  and  potential  of  the  Indi¬ 
vidual  Army  officer.  Immediate  objectives  of  the  research  are  to  de¬ 
termine  acceptability  and  effectiveness  of  peer  ratings  in  the  colonel 
grade  as  a  source  of  unique  information  In  Identifying  those  individuals 
with  the  highest  potential  for  performance  as  general  officers. 

b.  The  procedures  to  be  followed  are  designed  to  maintain  full 
confidentiality  of  the  rating  process.  The  mark-sense  rating  forms 
which  are  to  be  completed  are  prepared  for  machine  processing  and  upon 
return  will  be  converted  to  punch-card  form.  (These  rating  forms  are 
serially  pre-coded  for  administrative  control  of  returns,  and  for  MOS- 
career  field  and  other  categorization  in  the  data  analysis.  No  other 
uses  will  be  made  of  this  pre-code,  and  it  will  not  be  further  identi¬ 
fied  with  the  individual  rater.) 

c.  In  the  present  research  phase,  no  results  will  be  made  available 
for  inclusion  in  the  officer's  official  file  or  for  personnel  management 
decisions,  and  results  will  not  be  made  known  to  the  rated  officers. 
However,  to  maximize  the  utility  of  this  research  and  its  value  in  deci¬ 
sions  concerning  subsequent  implementation,  each  participant  is  urged  to 
consider  all  of  his  responses  as  if  intended  for  operational  use. 

d.  Ratings  are  to  be  accomplished  within  five  working  days  of 
receipt  and  completed  rating  forms  mailed  direct  to  the  United  States 
Army  Behavior  and  Systems  Research  Laboratory  in  the  inclosed  return 
envelope  for  analysis  and  compilation  of  summary  results. 

2.  Materials  Required 

a.  This  manual,  PT  4789,  including  the  rosters  of  officers  to  be 

considered  in  the  ratings. 

b.  Rating-Response  Forms  Set  comprised  of: 

(1)  Associate  Rating  of  Senior  Officer  Potential.  Part  A — 
Rating  Form,  PT  4790-1  thru  4790-3 

(2)  Part  B — Response  Form,  FT  4790a 

(3)  Evaluation  Form,  Evaluation  of  Associate  Rating  Program. 

PT  4790b 
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c.  Franked,  addressed  return  envelope 

d.  Number  2  lead  pencils 


3.  Officers  to  Be  Rated 

Offlcera  presently  serving  In  the  grade  of  colonel,  who  were  selec¬ 
ted  for  AUS-APL  promotion  to  colonel  in  FY  -  1966,  as  listed  on  the 
enclosed  roster. 


4.  Officers  to  Accomplish  the  Ratings 

The  officers  who  are  themselves  to  be  rated,  and  those  officers 
presently  serving  In  the  grade  of  colonel  who  were  selected  for  AUS-APL 
promotion  to  colonel  in  FY  1967* 


3.  The  Rating  Process 

a.  Ratings  Within  the  Rater's  Own  Career  Branch 

(1)  Consider  all  of  the  officers  within  your  own  career  branch 
as  listed  on  the  roster  which  follows,  and  underline  the  names  of  all 
the  officers  that  you  know  well  enough  to  form  a  judgment  about  the 
quality  of  their  past  performance  and  potential  for  future  assignments — 
officers  that  you  have  recently  served  with  or  otherwise  have  evidence 
of  their  performance  in  the  past. 

(2)  Then,  considering  the  relative  potential  of  each  of  the 
officers  whose  names  you  have  underlined,  select  those  that  you  consider 
to  have  the  highest  potential  to  perform  as  general  officers  and  those 
that  you  consider  to  have  the ‘least  potential  to  perform  as  general 
officers.  The  number  of  officers  you  are  to  select  for  each  category 

is  determined  by  the  number  of  officers  in  the  career  branch  as  listed 
on  the  roster.  Use  the  following  guide  to  determine  the  number  to 
include  in  each  category: 

(a)  Career  branch  roster  of  fifty  or  more  officers. 

Choose  at  least  five  and  not  more  than  ten  officers 
for  each  category  -  the  highest  and  the  least. 

(b)  Career  branch  rosters  of  at  least  twenty  five  but 
less  than  fifty  officers.  Choose  five  officers  for 
each  category  -  the  highest  and  the  least. 

(c)  Career  branch  roster  of  at  least  ten  but  less  than 
twenty  five  officers.  Choose  three  officers  for  each 
category  -  the  highest  and  the  least. 
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(d)  Career  branch  roster  o£  less  than  ten  officers.  Choose 
about  one  half  of  the  officers  for  each  category  -  the 
highest  and  the  least. 

(NOTE:  If  the  number  of  names  underlined  Is  less  than  the  number 
determined  In  accordance  with  these  guides,  each  of  the 
underlined  names  Is  to  be  placed  In  a  category  -  the  highest 
or  the  least.) 


(3)  On  the  roster,  circle  the  "man  number"  for  each  of  the 
officers  that  you  have  selected  In  accordance  with  the  procedures  out* 
lined  In  the  preceding  subparagraph,  Identifying  your  choices  of 
highest  and  least  potential. 


(4)  Carefully  fold  open  the  Rating-Response  Forms  Set  at  the 
tacked  corner  without  separating.  Place  the  appropriate  sheet  from 
Part  A  of  the  rating  form.  Associate  Rating  of  Senior  Officer  Potential 
(PT  4790-I  thru  -3)  on  the  roster  so  that  the  appropriate  career  branch 
section  of  the  form  is  in  alignment  with  the  same  portion  of  the  roster. 
Check  that  the  man  number  identification  on  the  roster  and  the  space 
number  on  the  rating  form  are  in  correspondence.  Then,  transfer  your 
choices  as  made  on  the  roster  to  the  rating  form  by  completely  blacken¬ 
ing  the  space  within  the  box  (using  a  number  2  pencil)  in  either  the 
"H"  (highest)  or  "L"  (least)  column  corresponding  to  your  choice.  Check 
the  accuracy  of  your  transfer  by  comparing  the  "man  number"  Identity  on 
both  the  roster  and  the  rating  form.  Avoid  making  any  stray  pencil 
marks  on  the  form  and  be  sure  that  all  erasures  are  complete  and  clean. 

(5)  Then,  for  each  officer  that  you  have  selected  in  accordance 

with  these  procedures,  indicate  the  degree  of  confidence  you  have  in 
your  choice,  by  marking  In  the  appropriate  column  ("M",  or  "L")  for 

each  choice  in  accordance  with  the  following: 

a.  "M"  -  Much  confidence,  well  acquainted  with  the  officer, 
and  have  solid  judgment  of  his  potential. 

"S"  -  Some  confidence,  occasional  contact  with  the 
officer,  and  have  a  general  opinion  of  his 
potential. 

"L"  -  Limited  confidence,  little  or  no  direct  contact 

with  the  officer,  primary  reliance  on  his  service 
reputation 
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b.  Ratings,  All  Career  Branches  Combined,  Except  Career  Branch  of  Rater 

(1)  Consider  all  officers  listed  on  the  roster,  except  those 
of  your  own  career  branch,  who  are  graduates  (or  students)  of  one  of 
the  senior  service  colleges  (identified  by  an  asterisk  with  the  name  on 
the  roster).  Underline  the  names  of  those  that  you  know  well  enough  to 
form  a  judgment  about  the  quality  of  their  past  performance  and  estimate 
of  their  potential  for  performance  in  the  general  officer  grade. 

(2)  Then,  from  the  underlined  names,  select  at  least  five  but 
not  more  than  ten  officers  with  the  highest  general  officer  potential, 
and  at  least  five  but  not  more  than  ten  officers  with  the  least  general 
officer  potential.  Circle  the  man  number  on  the  roster  for  these  indi¬ 
viduals,  and  then  transcribe  these  choices  to  the  corresponding  "man 
number"  spaces  on  the  rating  form  in  the  same  manner  as  described  above 
for  the  rating  of  officers  within  your  own  career  branch,  checking  care¬ 
fully  on  the  accuracy  of  this  transcription.  Then,  indicate  the  degree 
of  confidence  in  these  choices,  following  the  same  procedures  as  for  the 
selections  you  made  within  your  own  career  branch. 

c.  Justification  of  Rating  Choices  (Optional) 

Part  B  of  the  "Associate  Rating  of  Senior  Officer  Potential" 
form  provides  space  for  optional  entry  of  any  explanatory  or  clarify¬ 
ing  remarks  concerning  performance,  training  or  other  noteworthy  char¬ 
acteristics,  which  you  may  wish  to  include  about  any  of  the  individual 
officers  that  you  have  nominated. 


6.  Evaluation 

After  you  have  completed  the  rating  process,  as  defined  above,  you 
are  requested  to  express  your  reaction  to  the  program  and  its  accept¬ 
ability  and  potential  utility  for  operational  use  by  answering  the  ques¬ 
tions  on  the  Evaluation/Comment  Form — Evaluation  of  Associate  Rating 
Program.  FT  4790b. 


J.  Disposition  of  Materials 

This  manual  and  the  forms,  "Associate  Rating  of  Senior  Officer 
Potential"  and  "Evaluation  of  Associate  Rating  Program",  are  to  be 
mailed,  unfolded,  in  the  enclosed  envelope  as  soon  as  completed. 
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Evaluation  of  Associate  Rating  Program 
Evaluotion/Comment  Form 


Based  on  your  reaction  in  completing  the  subject  rating  program, 
your  opinions  and  attitudes  concerning  the  utility  in  specific  opera¬ 
tional  situations  are  requested.  Answer  each  of  the  following  ques¬ 
tions  fully  and  frankly.  (Use  a  number  two  pencil  to  nark  the  appro¬ 
priate  space  for  those  questions  which  require  selection  of  an  alter- 
na  t  i  ve . 

1.  If  an  index  score  (based  on  the  proportion  of  "high"  nominations 
and  "le.-st"  nominations  for  the  given  ratee)  were  computed  for  each 
officer  in  the  group  you  have  just  considered,  would  you  favor  it 
being  mode  available  for  use: 

a.  in  Promotion  Board  considerations? 

b.  by  Promotion  Approving  Authorities  only? 

c.  in  the  selection  board  consideration  for  senior  service 

college? 

d.  by  personnel  managers  in  making  duty  assignments? 


COMMENTS 


2.  Would  you  favor  operational  use  of  such  a  program  in  the  lower 
officer  grades? 

If  so,  at  what  grades?  (More  than  ore  r-.ay  be 

T .  Would  you  favor  such  a  rating  program  at  regular  intervals  in  each 
officer's  career? 

r 

If  so,  at  what  intervals?  Every  '  '  years. 

beginning  at  which  year  of  csvari ss i mec  service? 
t- .  General  Comments: 
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